# README for When Student Incentives Don't Work: Evidence from a Field Experiment in Malawi

Overview
--------
The code in this replication package constructs the analysis file from the three data sources using STATA. All data sources are primary data, as the authors collected the data in collaboration with AFF. One master file runs all of the code to generate the data for the 4 figures and 20 tables in the paper. The replicator should expect the code to run for about 4 minutes.

Data Availability and Provenance Statements
----------------------------
- There are multiple datasets included in the package: Survey datasets and exam score datasets.
- Survey questionnaires used to collect the data are included in the package.
- The evaluations are described on the projects Social Science Registry website: https://www.socialscienceregistry.org/trials/1119

### Statement about Rights

- [x] I certify that the author(s) of the manuscript have legitimate access to and permission to use the data used in this manuscript. 

### Summary of Availability
- [x] All data **are** publicly available.
- [ ] Some data **cannot be made** publicly available.
- [ ] **No data can be made** publicly available.


Dataset list
------------
- All datasets are provided in .dta format
- Variables are labelled. If not, a questionnaire used to collect the data (where variable names are linked to) are provided in the package.
- Several variables are dropped from the original dataset to prevent revealing the identity of an individual.

| Dataset            | Description                                           |
|--------------------|-------------------------------------------------------|
| y1_raw.dta         | Baseline and follow-up survey data + final exam score |
| y2_raw.dta         | longer-term sample survey data + exam score |
| score_baseline.dta | baseline exam score                                   |
| score_midline.dta  | midline exam score                                    |

Structure of the dataset
----------------
Before you begin replication, please recover the following directory structure.
- Root directory: Replication_package
    - README.pdf and README.md should be saved here

- Subdirectories: code, data, document
    - code: all .do files must be saved here.
    - data: all .dta files must be saved under the subdirectory `data/raw`.
    - document: `Baseline_Stuent_Final.pdf`, `Followup_Student_Final.pdf`, and `Longer_Baseline_Student_Final.pdf` must be saved here.


Computational requirements
---------------------------

### Software Requirements
- Stata (code was last run with version 16)
  - `estout`
  - `coefplot`
  - the program `config.do` will install all dependencies locally, and should be run once.
  - user-written programs are defined in `programs.do`


### Memory and Runtime Requirements

#### Summary
Approximate time needed to reproduce the analyses on a standard (CURRENT YEAR) desktop machine:
- [x] <10 minutes

#### Details
The code was last run on a **CISER Shared Windows Server 2019, 256GB, Intel Xeon E5-4669 v3 @ 2.10Ghz (2 processors, 36 cores)**. 


Description of programs/code
----------------------------
- All programs are stored in a directory named "code".
- `malawi_student_incentive_master` will run all programs in sequence to clean dataset and generate all tables and figures. 
- `config.do` contains parameters used by all programs. It also starts writing the log file and install necessary programs to run the program. If running in any order other than the one outlined below, your results may differ.
- `programs` contains user-written programs used in the analyses.
- `malawi_student_incentive_variable_generation` prepares the raw datasets into a analysis datafile, `merged_y1y2.dta`.
- `malawi_student_incentive_table` generate all tables in the main body of the article. 
- `malawi_student_incentive_graph` generate all figures in the main body of the article. 


Instructions to Replicators
---------------------------
- Edit `malawi_student_incentive_master.do` to adjust the default path
- Run `config.do` once on a new system to set up the working environment. 
- Run `malawi_student_incentive_master.do` to run all steps in sequence.

List of tables and programs
---------------------------
The provided code reproduces:

- [x] All numbers provided in text in the paper
- [x] All tables and figures in the paper
- [ ] Selected tables and figures in the paper, as explained and justified below.

- Figures and Tables in the Main text

| Figure/Table #                      | Programs                       | Line number      | output file                                                                              |
|-------------------------------------|--------------------------------|------------------|------------------------------------------------------------------------------------------|
| Table 1                             | malawi_student_incentive_table | 151-229          | sample_comp1.tex                                                                         |
| Table 2                             | malawi_student_incentive_table | 232-344          | balance.tex                                                                              |
| Table 3                             | malawi_student_incentive_table | 486-541          | s_expectation.tex                                                                        |
| Table 4                             | malawi_student_incentive_table | 568-673          | s_exam.tex                                                                               |
| Table 5                             | malawi_student_incentive_table | 737-913          | s_extm_dist.tex                                                                          |
| Table 6                             | malawi_student_incentive_table | 915-979          | s_intermediate.tex                                                                       |
| Table 7                             | malawi_student_incentive_table | 679-697          | s_exam_long.tex                                                                          |
| Figure 1                            | Not empirical                  |                  | experimentaldesign.tex                                                                   |
| Figure 2                            | Not empirical                  |                  | s_rand_ann.tex                                                                           |
| Figure 3                            | malawi_student_incentive_graph | 102-129          | understand_pct_58.png                                                                    |
| Figure 4                            | malawi_student_incentive_graph | 130-136, 140-166 | expect_pct_58.png, expect_sub_58.png                                                     |
| Figure 5                            | malawi_student_incentive_graph | 176-213          | s_exam_58, s_exam_sub_58.png                                                             |

- Figures and Tables in the Appendix

| Figure/Table #                      | Programs                       | Line number      | output file                                                                              |
|-------------------------------------|--------------------------------|------------------|------------------------------------------------------------------------------------------|
| Table A1                            | malawi_student_incentive_table | 1269-1337        | attrition.tex                                                                            |
| Table A2                            | malawi_student_incentive_table | 1390-1446        | s_exam_subject.tex                                                                       |
| Table A3                            | malawi_student_incentive_table | 1476-1581        | s_exam_gender.tex                                                                        |
| Table A4                            | malawi_student_incentive_table | 1014-1060        | s_intermediate_long.tex                                                                  |
| Table A5                            | malawi_student_incentive_table | 1583-1655        | app_exam_intermediate.tex                                                                |
| Table A6                            | malawi_student_incentive_table | 1672-1726        | s_exam_standards.tex                                                                     |
| Table A7                            | malawi_student_incentive_table | 1743-1798        | app_class_environment.tex                                                                |
| Figure A1                           | Not empirical                  |                  | quiz1.png                                                                                |
| Figure A2                           | Not empirical                  |                  | quiz2.png                                                                                |
| Table B1                            | malawi_student_incentive_table | 1829-1950        | feedback_balance.tex                                                                     |
| Table B2                            | malawi_student_incentive_table | 1339-1358        | attrition_feedback.tex                                                                   |
| Table B3                            | malawi_student_incentive_table | 1955-2016        | f_exam.tex                                                                               |
| Table B4                            | malawi_student_incentive_table | 2019-2068        | f_additional.tex                                                                         |
| Table B5                            | malawi_student_incentive_table | 2071-2140        | s_exam_a.tex                                                                             |
| Figure B1                           | Not empirical                  |                  | feedback1.png, feedback2.png, feedback3.png, feedback4.png, feedback5.png, feedback6.png |
| Figure B2                           | malawi_student_incentive_graph | 230-273          | f_exam.png, f_exam_pct_58.png                                                            |
| Table C1                            | lee bounds working.do          | 64-170           | leebounds.tex                                                                            |
| Table C2                            | lee bounds working.do          | 174-220          | attrit_interact.tex                                                                      |